Sequence-to-sequence models have been applied to the conversation responsegeneration problem where the source sequence is the conversation history andthe target sequence is the response. Unlike translation, conversationresponding is inherently creative. The generation of long, informative,coherent, and diverse responses remains a hard task. In this work, we focus onthe single turn setting. We add self-attention to the decoder to maintaincoherence in longer responses, and we propose a practical approach, called theglimpse-model, for scaling to large datasets. We introduce a stochasticbeam-search algorithm with segment-by-segment reranking which lets us injectdiversity earlier in the generation process. We trained on a combined data setof over 2.3B conversation messages mined from the web. In human evaluationstudies, our method produces longer responses overall, with a higher proportionrated as acceptable and excellent as length increases, compared to baselinesequence-to-sequence models with explicit length-promotion. A back-off strategyproduces better responses overall, in the full spectrum of lengths.
展开▼
